
<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

    <title>State machine &#8212; LAVA 2024.05 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/bootstrap-sphinx.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
    <script src="_static/jquery.js"></script>
    <script src="_static/underscore.js"></script>
    <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
    <script src="_static/doctools.js"></script>
    <script src="_static/sphinx_highlight.js"></script>
    <link rel="shortcut icon" href="_static/favicon.ico"/>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="next" title="Functional testing of LAVA source code" href="functional_tests.html" />
    <link rel="prev" title="Developing new classes for LAVA V2" href="developer-example.html" />
    <link rel="canonical" href="https://docs.lavasoftware.org/lava/scheduler.html" />
  
<meta charset='utf-8'>
<meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'>
<meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1'>
<meta name="apple-mobile-web-app-capable" content="yes">
<script type="text/javascript" src="_static/js/jquery-1.12.4.min.js"></script>
<script type="text/javascript" src="_static/js/jquery-fix.js"></script>
<script type="text/javascript" src="_static/bootstrap-3.4.1/js/bootstrap.min.js"></script>
<script type="text/javascript" src="_static/bootstrap-sphinx.js"></script>


  </head><body>

  <div id="navbar" class="navbar navbar-default navbar-fixed-top">
    <div class="container">
      <div class="navbar-header">
        <!-- .btn-navbar is used as the toggle for collapsed navbar content -->
        <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse">
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
        </button>
        <a class="navbar-brand" href="index.html"><span><img src="_static/lava.png"></span>
          LAVA</a>
        <span class="navbar-text navbar-version pull-left"><b>2024.05</b></span>
      </div>

        <div class="collapse navbar-collapse nav-collapse">
          <ul class="nav navbar-nav">
            
                <li><a href="genindex.html">Index</a></li>
                <li><a href="contents.html">Contents</a></li>
            
            
              <li class="dropdown globaltoc-container">
  <a role="button"
     id="dLabelGlobalToc"
     data-toggle="dropdown"
     data-target="#"
     href="index.html">Site <b class="caret"></b></a>
  <ul class="dropdown-menu globaltoc"
      role="menu"
      aria-labelledby="dLabelGlobalToc"><ul class="current">
<li class="toctree-l1"><a class="reference internal" href="index.html">Introduction to LAVA</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="contents.html">Contents</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="glossary.html">Glossary of terms</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="support.html">Getting support</a></li>
</ul>
</ul>
</li>
              
                <li class="dropdown">
  <a role="button"
     id="dLabelLocalToc"
     data-toggle="dropdown"
     data-target="#"
     href="#">Page <b class="caret"></b></a>
  <ul class="dropdown-menu localtoc"
      role="menu"
      aria-labelledby="dLabelLocalToc"><ul>
<li><a class="reference internal" href="#">State machine</a><ul>
<li><a class="reference internal" href="#workers">Workers</a></li>
<li><a class="reference internal" href="#devices">Devices</a></li>
<li><a class="reference internal" href="#testjobs">TestJobs</a></li>
</ul>
</li>
<li><a class="reference internal" href="#scheduler">Scheduler</a><ul>
<li><a class="reference internal" href="#health-checks">Health-checks</a></li>
<li><a class="reference internal" href="#test-jobs">Test jobs</a></li>
</ul>
</li>
</ul>
</ul>
</li>
              
            
            
              
                
  <li>
    <a href="developer-example.html" title="Previous Chapter: Developing new classes for LAVA V2"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">&laquo; Developing ne...</span>
    </a>
  </li>
  <li>
    <a href="functional_tests.html" title="Next Chapter: Functional testing of LAVA source code"><span class="glyphicon glyphicon-chevron-right visible-sm"></span><span class="hidden-sm hidden-tablet">Functional te... &raquo;</span>
    </a>
  </li>
              
            
            
            
            
              <li class="hidden-sm"></li>
            
          </ul>

          
            
<form class="navbar-form navbar-right" action="search.html" method="get">
 <div class="form-group">
  <input type="text" name="q" class="form-control" placeholder="Search" />
 </div>
  <input type="hidden" name="check_keywords" value="yes" />
  <input type="hidden" name="area" value="default" />
</form>
          
        </div>
    </div>
  </div>

<div class="container">
  <div class="row">
    <div class="body col-md-12 content" role="main">
      
  <section id="state-machine">
<span id="index-0"></span><span id="id1"></span><h1>State machine<a class="headerlink" href="#state-machine" title="Permalink to this heading">¶</a></h1>
<p>The state machine describes and controls the state and health of Workers,
Devices and Test Jobs.</p>
<section id="workers">
<h2>Workers<a class="headerlink" href="#workers" title="Permalink to this heading">¶</a></h2>
<p>For each worker, two variables describe the current status:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">state</span></code>:</p>
<ul>
<li><p><em>Online</em></p></li>
<li><p><em>Offline</em></p></li>
</ul>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">health</span></code>:</p>
<ul>
<li><p><em>Active</em></p></li>
<li><p><em>Maintenance</em></p></li>
<li><p><em>Retired</em></p></li>
</ul>
</li>
</ul>
<p><code class="docutils literal notranslate"><span class="pre">state</span></code> is an internal variable, set by <code class="docutils literal notranslate"><span class="pre">lava-master</span></code> when receiving (or
not) pings from each worker.</p>
<div class="admonition caution">
<p class="admonition-title">Caution</p>
<p>When a worker is in <em>Offline</em>, none of the attached devices
will be used to schedule new jobs.</p>
</div>
<p><code class="docutils literal notranslate"><span class="pre">health</span></code> can be used by admins to prevent jobs to run on attached devices.
For instance, when set to <em>Maintenance</em>, no jobs will be scheduled on attached
devices.</p>
</section>
<section id="devices">
<h2>Devices<a class="headerlink" href="#devices" title="Permalink to this heading">¶</a></h2>
<p>For each device, two variables describe the current status:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">state</span></code>:</p>
<ul>
<li><p><em>Idle</em>: not in use by any test job</p></li>
<li><p><em>Reserved</em>: has been reserved for a test job but the test job is not
running yet</p></li>
<li><p><em>Running</em>: currently running a test job</p></li>
</ul>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">health</span></code>:</p>
<ul>
<li><p><em>Good</em></p></li>
<li><p><em>Unknown</em></p></li>
<li><p><em>Looping</em>: should run health-checks in a loop</p></li>
<li><p><em>Bad</em></p></li>
<li><p><em>Maintenance</em></p></li>
<li><p><em>Retired</em></p></li>
</ul>
</li>
</ul>
<p><code class="docutils literal notranslate"><span class="pre">state</span></code> is an internal variable, set by <code class="docutils literal notranslate"><span class="pre">lava-server-gunicorn</span></code> and <code class="docutils literal notranslate"><span class="pre">lava-scheduler</span></code>
when scheduling, starting, canceling and ending test jobs.</p>
<p><code class="docutils literal notranslate"><span class="pre">health</span></code> can be used by admins to indicate if a device should be used by the
scheduler or not. Moreover, when ending an health-check, the device health will
be set according to the test job health.</p>
</section>
<section id="testjobs">
<h2>TestJobs<a class="headerlink" href="#testjobs" title="Permalink to this heading">¶</a></h2>
<p>For each test job, two variables are describing the current status:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">state</span></code>:</p>
<ul>
<li><p><em>Submitted</em>: waiting in the queue</p></li>
<li><p><em>Scheduling: part of a multinode test job where some sub-jobs are
still in *Submitted</em></p></li>
<li><p><em>Scheduled</em>: has been scheduled. For multinode, it means that all
sub-jobs are also scheduled</p></li>
<li><p><em>Running</em>: currently running on a device</p></li>
<li><p><em>Canceling</em>: has been canceled but not ended yet</p></li>
<li><p><em>Finished</em></p></li>
</ul>
</li>
</ul>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Only multinode test jobs use <em>Scheduling</em>. When all
sub-jobs are in <em>Scheduling</em>, <code class="docutils literal notranslate"><span class="pre">lava-master</span></code> will transition all test
jobs to <em>Scheduled</em>.</p>
</div>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">health</span></code>:</p>
<ul>
<li><p><em>Unknown</em>: default value that will be overridden when the job is finished.</p></li>
<li><p><em>Complete</em></p></li>
<li><p><em>Incomplete</em></p></li>
<li><p><em>Canceled</em>: the test job was canceled.</p></li>
</ul>
</li>
</ul>
</section>
</section>
<section id="scheduler">
<span id="id2"></span><h1>Scheduler<a class="headerlink" href="#scheduler" title="Permalink to this heading">¶</a></h1>
<p>The scheduler is called by <code class="docutils literal notranslate"><span class="pre">lava-master</span></code> approximately every 20 seconds.
The scheduler starts by scheduling health-checks. The remaining devices are
then considered for test jobs.</p>
<section id="health-checks">
<h2>Health-checks<a class="headerlink" href="#health-checks" title="Permalink to this heading">¶</a></h2>
<p>To ensure that health-checks are always scheduled when needed, they will be
considered first by the scheduler before regular test jobs.</p>
<p>The scheduler will only consider devices where:</p>
<ul class="simple">
<li><p><cite>state</cite> is <em>Idle</em></p></li>
<li><p><cite>health</cite> is <em>Good</em>, <em>Unknown</em> or <em>Looping</em></p></li>
<li><p>worker’s <cite>state</cite> is <em>Online</em></p></li>
</ul>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>A device whose <code class="docutils literal notranslate"><span class="pre">health</span></code> is <em>Bad</em>, <em>Maintenance</em> or <em>Retired</em> is
never considered by the scheduler when it is looking for devices to run test
jobs</p>
</div>
</section>
<section id="test-jobs">
<h2>Test jobs<a class="headerlink" href="#test-jobs" title="Permalink to this heading">¶</a></h2>
<p>The scheduler will only consider devices where:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">state</span></code> is <em>Idle</em></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">health</span></code> is <em>Good</em> or <em>Unknown</em></p></li>
<li><p>worker’s <code class="docutils literal notranslate"><span class="pre">state</span></code> is <em>Online</em></p></li>
</ul>
</section>
</section>


    </div>
      
  </div>
</div>
<footer class="footer">
  <div class="container">
    <p class="pull-right">
      <a href="#">Back to top</a>
      
    </p>
    <p>
        &copy; Copyright 2010-2019, Linaro Limited.<br/>
      Created using <a href="http://sphinx-doc.org/">Sphinx</a> 5.3.0.<br/>
    </p>
  </div>
</footer>
  </body>
</html>